Super-resolution Mosaicing for Object based Video Format Conversion

This paper presents a new approach to spatial upsampling of digital video based on super-resolution mosaics. First, we robustly generate a background mosaic of higher resolution than the original video. In order to achieve that goal, we apply hierarchical global image registration estimating an optimal parabolic parameter set for each view of a scene shot. The final mosaic is generated using statistical and projection grid distance measures to avoid the impact of foreground objects and to accomplish super-resolution respectively. Second, arbitrarily moving foreground objects are segmented using MRF-based change detection methods based on the calculated mosaic. For the foreground objects an optical flow field between adjacent frames is computed. Third, we create new views with higher spatial resolution fusing re-projected background content from the mosaic together with super-resolution foreground objects obtained using optical flow field calculation. Results show that this method is able to convert videos into higher spatial resolution with very high objective and subjective quality.